Managing servers with quality of service assurances

ABSTRACT

Aspects of an embodiment of the invention disclose a method, computer program product, and system for managing the energy efficiency of servers providing multi-class computing services with Quality of Service (QoS) assurance. Computing resources are clustered into at least three groups, where each group has a separate power management policy (PMP). A plurality of requests are received from a plurality of devices, and are sorted into at least three service classes based on the requests&#39; QoS criteria. Each request is assigned to one of at least three service queues based on the request&#39;s service class, and each service group is processed by a group of computing resources. The power management policies are configured such that each group of computing resources may service requests at an energy efficient point while meeting the QoS criteria of the service class.

BACKGROUND

The present disclosure relates generally to the field of servercomputing, and more particularly to managing the energy efficiency ofservers providing multi-class computing services.

Server admins have a number of resources available to balance the energyefficiency of a server with the processing capabilities. Server adminsmay set power management policies to match processing requirements, andthe policies in turn may use features such as dynamic voltage andfrequency scaling (DVFS) as to raise or lower the voltage and frequencyof a server's processor to adjust power consumption. Modern processorsmay also have low power modes, such as nap or sleep modes, which can beutilized to decrease the energy demand when the server is at idle.Additionally, many modern processors are multi-core processors that canaccommodate consolidation of work onto a subset of the cores. This maybe called core folding or core parking or other names that convey theconsolidation of work. Core folding policy, when enabled, allows theoperating system or privileged system software to consolidate theserver's work into fewer cores, and unused cores can be turned off orplaced in a low power mode to decrease the server's energy demand.

Modern mobile devices, such as smart phones and tablets, haveapplications with a wide range of Quality of Service (QoS) requirements.Some applications, including online gaming and video streaming, mayrequire an almost instantaneous response. Other applications, likeprinting services and file downloads, may not be processed by a serverfor several seconds with little to no impact on the user's experience.

SUMMARY

Aspects of an embodiment of the invention disclose a method for managingthe energy efficiency of servers providing multi-class computingservices with Quality of Service (QoS) assurance. The method comprisesreceiving a plurality of requests from a plurality of devices, sortingthe requests into at least three service classes based on the requests'QoS criteria, and assigning each request to one of at least threeservice queues based on the request's service class. The method furthercomprises grouping computing resources into at least three groups, whereeach group has a separate power management policy (PMP) and handles atleast one service queue. The power management policies are configuredsuch that each group of computing resources may service requests at anenergy efficient point while meeting the QoS criteria of the serviceclass.

Additional embodiments of the present disclosure are directed to asystem and a computer program product for managing the energy efficiencyof servers providing multi-class computing services with QoS assurance.

The above summary is not intended to describe each illustratedembodiment or every implementation of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings included in the present application are incorporated into,and form part of, the specification. They illustrate embodiments of thepresent invention and, along with the description, serve to explain theprinciples of the invention. The drawings are only illustrative oftypical embodiments of the invention and do not limit the invention.

FIG. 1 depicts a cloud computing node, in accordance with an embodimentof the present disclosure.

FIG. 2 depicts a cloud computing environment, in accordance with anembodiment of the present disclosure.

FIG. 3 depicts abstraction model layers, in accordance with anembodiment of the present disclosure.

FIG. 4 is a block diagram of exemplary components that may be used inimplementing one or more of the methods, tools, and modules, and anyrelated functions, described herein, in accordance with an embodiment ofthe present disclosure.

FIG. 5 is a flowchart illustrating a method for assigning incomingrequests to service queues based on the requests' Quality of Service(QoS) criteria, in accordance with an embodiment of the presentdisclosure.

FIG. 6 is a flowchart illustrating a method for adjusting powermanagement policies (PMPs) to manage the energy use in a server with QoSassurance, in accordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION

The present disclosure relates generally to the field of servercomputing, and more particularly to managing the energy efficiency ofservers providing multi-class computing services. While the presentdisclosure is not necessarily limited to such applications, variousaspects of the disclosure may be appreciated through a discussion ofvarious examples using this context.

As used herein, a “device” is any object or machine that may communicatewith, and send requests to, a front-end server. Devices may be, e.g.,smart phones, tablets, desktop computers, laptop computers, and gamingmachines. A “front end server” is any server or software applicationthat can receive incoming requests, sort them into classes based ontheir Quality of Service (QoS) criteria, and assign them to a servicequeue. A front end server may be a physical or virtual interface betweenthe devices and the computing resources. The duties of the front endserver may be distributed amongst one or more systems. The front endserver may be, e.g., a software application running on a computersystem.

Class of Service (CoS) is a way of managing traffic in a network bygrouping similar types of traffic (for example, e-mail, streaming video,voice, large document file transfer) together and treating each type asa class with its own level of service priority. A “service class” is oneof those groups. “Service queue information” is any information about aservice queue that may be useful in determining whether the back-endservers are meeting the QoS criteria of the jobs. For example, servicequeue information may include the queue depth (the number of jobs in thequeue), the arrival rate (rate at which new jobs are being assigned tothe queue), and the service rate (rate at which jobs in the queue arebeing processed). A “back-end server” is a group of computing resourcesassigned to one or more service queues. “Computing resources” areprocessing units that may handle a service queue. Computing resourcesmay be, e.g., processor cores, central processing units (CPUs), orservers.

A group of computing resources may be operating at “an energy efficientpoint” when the ratio of its performance of executing tasks to its powerconsumption is above a predetermined efficiency threshold. In someembodiments, the efficiency threshold may be set by a server admin asthe minimum acceptable performance to power ratio for the computingresources. For example, the efficiency threshold may be 5% below themaximum performance to power consumption ratio of the computingresources.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 1, a schematic of an example of a cloud computingnode is shown. Cloud computing node 10 is only one example of a suitablecloud computing node and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the inventiondescribed herein. Regardless, cloud computing node 10 is capable ofbeing implemented and/or performing any of the functionality set forthhereinabove.

In cloud computing node 10 there is a computer system/server 12, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 1, computer system/server 12 in cloud computing node 10is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 12 may include, but are not limitedto, one or more processors or processing units 16, a system memory 28,and a bus 18 that couples various system components including systemmemory 28 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

For example, in an embodiment of the present disclosure, the programmodules 42 may include a service queue module, a QoS monitoring module,and a resource and energy management module. The service queue modulemay include instructions to determine the number of service queuesactive in the system, receive a request from a device, determine therequest's service class based on its QoS criteria, and assign therequest to the appropriate service queue. The QoS monitoring module mayinclude instructions to monitor the queue depth of each service queue,and instructions to determine whether the back-end servers areprocessing the requests fast enough to meet the QoS criteria. Theresource and energy management module may include instructions to adjustthe power management policies (PMPs) of the back-end servers to ensurethat the back-end servers are meeting the QoS criteria of the servicequeues to which they are assigned, and instructions to ensure that theback-end servers are running at an energy efficient point given the QoScriteria.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

It is noted that FIG. 1 is intended to depict the representative majorcomponents of an exemplary computer system/server 12. In someembodiments, however, individual components may have greater or lessercomplexity than as represented in FIG. 1, components other than or inaddition to those shown in FIG. 1 may be present, and the number, type,and configuration of such components may vary.

Referring now to FIG. 2, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 3, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 2) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 3 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and mobile desktop 96.

Referring now to FIG. 4, shown is a block diagram of exemplarycomponents that may be used in implementing one or more of the methods,tools, and modules, and any related functions, described herein, inaccordance with an embodiment of the present disclosure. In someembodiments, the major components may include one or more devices402A-C, collectively referred to as devices 402, a front-end server 404,a plurality of service queues 406, 408, and 410, a QoS monitor 412, aresource and energy manager 420, a plurality of power managementpolicies 415, 417, and 419, and a plurality of back-end servers 414,416, and 418.

In the depicted example, the devices 402 may be mobile devices thatcommunicate with the front-end server 404 through a telecommunicationsnetworks, e.g., cell phones. In some embodiments, the disclosure may beimplemented using a number of different types of communication networks,such as the Internet, an intranet, a local area network (LAN), or a widearea network (WAN). In some embodiments, the devices 402 may include,for example, a cellular telephone, a desktop computer, a laptopcomputer, a handheld computer, a tablet, a personal digital assistant,or a gaming device.

The devices 402 may connect to the front-end server using wiredcommunication links, wireless communication links, or fiber opticcables. Examples of wireless communication links include shortwave, highfrequency, ultra-high frequency, microwave, wireless fidelity (Wi-Fi),Bluetooth technology, global system for mobile communications (GSM),code division multiple access (CDMA), second-generation (2G),third-generation (3G), fourth-generation (4G), or any other wirelesscommunication technology or standard to establish a wirelesscommunications link. It should be noted that the devices 402 mayrepresent any combination of different devices connected to thefront-end server 404.

In some embodiments, such as the example depicted in FIG. 4, the presentdisclosure may be implemented using three service queues: a real-time(RT) service queue 406, a deadline (DL) service queue 408, and anavailable resources (AR) service queue 410. Requests arrive at the frontend server 404 and may be assigned to one of a plurality of serviceclasses based on their QoS criteria. For example, requests with highpriority QoS criteria may be assigned to a real-time service class andsent to the real-time service queue 406, where they will receive areal-time response. Real-time service class requests may be requeststhat require extremely low latency, such as those in support of onlinegaming, GPS services, mobile commerce, and video streaming. A real-timeresponse is a server response that ensures the back-end server meets theQoS criteria for the real-time service class.

Requests with medium priority QoS criteria, where a small but noticeablelatency is acceptable, e.g., web searching, may be assigned to adeadline service class and sent to the deadline service queue 408 andreceive a deadline response. A deadline response is a server responsethat ensures the back-end server meets the QoS criteria for the deadlineclass. Requests with low priority QoS criteria, such as file downloads,printing services, and background software updates, may be assigned toan available resources service class and may be sent to the availableresources service queue 410 where it may receive an available resourcesresponse. An available resources response is a server response thatensures the back-end server meets the QoS criteria for the availableresources service class. In some embodiments, each service queue mayhandle a single service class. In other embodiments, multiple serviceclasses may be assigned to a single service queue.

In some embodiments, more service queues may be utilized, and in someembodiments the service queues may be made up of two or more servicesub-queues. Each service queue may have an associated back-end server,such as, e.g., back-end servers 414, 416, and 418, that are responsiblefor handling the requests in the service queue. Each back-end server mayhave an associated power management policy (PMP) 415, 417, and 419 toensure the QoS criteria of the requests are met, and to ensure energyefficiency.

The PMPs 415, 417, and 419 are policies set by the resource and energymanager to ensure that QoS criteria are met for each service queue,while maintaining energy efficiency, especially for service queues thatdo not require a real-time response. The PMPs establish, e.g., thenumber of active processors and the frequency and voltage of theprocessors, for each back-end server. For example, in some embodimentsthe real-time PMP 415 may direct the real-time back-end server 414 toalways run all of its computing resources at their maximum frequency andvoltage. In other embodiments, the real-time PMP 415 may direct thereal-time back-end server 414 to always keep at least one core activeand, when a request arrives, run at its highest frequency and voltage.

The deadline PMP 417 may ensure power savings when compared to thereal-time PMP. The deadline PMP 417 may, in some embodiments, keep oneor more cores active at their minimum frequency (e.g. in standby mode)when no jobs are being processed to ensure power savings. When a jobarrives, the deadline PMP 417 may set the deadline back-end server 416to an energy efficient point that can satisfy the QoS criteria of thejob.

The available resources PMP 419 may ensure power savings when comparedto the deadline PMP. In some embodiments, the available resources PMP419 may keep one core in nap or sleep mode when no jobs are beinghandled to save power. When jobs arrive, the available resources PMP 419may activate as many cores of the available resources back-end server asis necessary to meet the requests' QoS criteria at an energy efficientpoint.

In some embodiments, a back-end server may be comprised of processingcores, i.e. cores within a processor. In other embodiments, the back-endservers may instead be one or more central processing units (CPUs). Inother embodiments, the back-end servers may be comprised of one or moreservers. In still other embodiments, the back-end server may becomprised of some combination of servers, CPUs, and processing cores.

A QoS monitor 412 is responsible for monitoring the queue depth of eachservice queue and determining whether each service queue's QoS criteriacan be met with the current PMP. If a back-end server is not going tomeet the QoS criteria of its service queue, the QoS monitor 412 maynotify a resource and energy manager 420. The resource and energymanager 420 is responsible for establishing and adjusting the PMPs foreach service queue.

The front-end server 404 is responsible for receiving requests from thedevices 402 and determining the requests' service class based on its QoScriteria. After determining the requests' service class, the front-endserver 404 assigns the requests to the appropriate service queues 414,416, or 418. The front-end server 404 may be a multi-user mainframecomputer system, a single-user system, or a server computer or similardevice that has little or no direct user interface, but receivesrequests from other computer systems (clients). Further, in someembodiments, the front-end server 404 may be implemented as a desktopcomputer, portable computer, laptop or notebook computer, tabletcomputer, pocket computer, smart phone, network switches or routers, orany other appropriate type of electronic device.

It is noted that FIG. 4 is intended to depict the representative majorcomponents of the present disclosure. In some embodiments, however,individual components may have greater or lesser complexity than asrepresented in FIG. 4, components other than or in addition to thoseshown in FIG. 4 may be present, and the number, type, and configurationof such components may vary. For example, in some embodiments, theback-end servers are subcomponents of a single server or mainframecomputer.

Referring now to FIG. 5, shown is a flowchart illustrating a method 500for assigning incoming requests to service queues based on the requests'QoS criteria, in accordance with an embodiment of the presentdisclosure. In some embodiments, the method 500 may be performed by anapplication running on a front end server 404 (shown in FIG. 4). Themethod may begin at operation 502, wherein the application may receivean incoming request from a device. At operation 504, the application maydetermine the number of service queues available or enabled in thesystem. In some embodiments, the number of service queues may be set bya user. In other embodiments, the number of service queues and servicesub-queues may dynamically adjust based on the depth of the servicequeues.

For example, a system could be configured to initially have threeservice queues: one for high priority (real-time class) requests, onefor medium priority (deadline class) requests, and one for low priority(available resources class) requests. If the service queue for themedium priority requests gets long because, e.g., an unusually highnumber of deadline class request arrive in a short period of time, theserver may respond by adjusting the PMP of the cores servicing thedeadline service queue. This may result in a less energy efficientprocessing of requests, however, because some deadline requests may beserviced faster than necessary in order to get through the backlog. Insuch circumstances, it may be advantageous to dynamically adjust thenumber of service queues by splitting the deadline service queue intotwo distinct sub-queues, each with its own PMP.

A first sub-queue may process deadline class requests that have beenwaiting longer or require a more immediate response. This firstsub-queue may have a first PMP that ensures a faster response than isnormal for deadline class requests. A second sub-queue may processdeadline class requests that do not require as immediate a response.These could be requests that were more recently added to the deadlineservice queue, or they may be requests that, while still in the deadlineclass, inherently require a less immediate response. The secondsub-queue may have a second PMP that is normal for, or even more energyefficient than, the normal deadline class PMP. By splitting the deadlineclass up, the server may be more energy efficient than it would be ifthe deadline class PMP were adjusted for all requests in the deadlineservice queue.

After determining the number of service queues at operation 504, theapplication may determine which service class the received requestbelongs in at operation 506. As discussed herein, the application mayrecognize three or more service classes. For example, in someembodiments, at operation 506 the received request may be grouped into ahigh priority (real-time) class, a medium priority (deadline) class, ora low priority (available resource) class, depending on the QoS criteriaof the request.

There are numerous ways in which the application may determine arequest's service class at operation 506. In some embodiments, theapplication may extract a request's header, which may contain anotification of the request's QoS criteria. In other embodiments, theapplication may look at the account information of the requestor whosent the request. If, for example, the requestor runs an online videogame, he may have an account set up so that all requests are given ahigh priority, real-time response. Many other methods for determiningthe appropriate service class exist, and the present disclosure shouldnot be limited to any particular method. After determining the request'sservice class, the application may assign the request to an appropriateservice queue at operation 508, and the method may end.

In some embodiments, all the operations in method 500 are performed bythe same application on the front-end server. In other embodiments, theoperations may be performed by two or more applications running on thefront-end server. In yet other embodiments, the operations in method 500may be performed by two or more servers.

Referring now to FIG. 6, shown is a flowchart illustrating a method 600for adjusting power management policies (PMPs) to manage the energy usein a server while meeting QoS criteria, in accordance with an embodimentof the present disclosure. In some embodiments, the method 600 may beperformed by an application running on the front end server 404 (shownin FIG. 4). The method 600 may begin with operation 602, wherein a QoSmonitor may receive service queue information.

Using the service queue information, the QoS monitor may determinewhether the back-end server handling the service queue will meet the QoScriteria at operation 604. In some embodiments, to determine whether theback-end server will meet the QoS criteria, the QoS monitor may comparethe current queue depth (the number of requests in the queue) to theservice rate (the rate at which requests are being serviced). In otherembodiments, the QoS monitor may check to see if any requests havemissed their response requirements. In other embodiments, the QoSmonitor may compare the queue depth to a maximum queue depth threshold,which is the maximum number of requests that may be in a queue.

In still other embodiments, the QoS monitor may compare the queue depth,the response time requirements of the service queue, and the arrivalrate of requests in the queue. Using Little's Law, the resource andenergy manager may determine the average response time of the queue.According to Little's Law, the average response time of the queue, R, isR=n/λ, where n is the queue depth and λ is the average rate of arrivalof new requests. The QoS monitor may then compare the average responsetime to the response time requirements of the service queue. If theaverage response time is lower, the QoS monitor may determine that QoScriteria are being met. If the average response time is higher, the QoSmonitor may determine that QoS criteria are not being met.

If the back-end server is not meeting the QoS criteria of the servicequeue, the resource and energy manager may adjust the back-end server'sPMP to increase the service rate at operation 608. There are a number ofways that the service rate may be increased. In some embodiments, theresource and energy manager may use Dynamic Voltage and FrequencyScaling (DVFS) to raise the frequency of the processors, or processorcores, in the back-end server handling the service queue. In someembodiments, additional cores may be activated (known as core unfolding)in the back-end server to increase the processing power.

In some embodiments, the PMP may be adjusted at operation 608 using DVFSto increase the frequency of the processing cores by a set amount. Inother embodiments, the resource and energy manager may predict thefrequency necessary to meet QoS criteria using, e.g., Little's Law.Then, the QoS monitor may determine whether the computing resources willmeet the QoS criteria per operation 604.

If the back-end server will meet the QoS criteria of the service queue,at operation 610 the resource and energy manager may determine if theback-end server is operating at an energy efficient point. To determineif the back-end server is operating at an energy efficient point, theresource and energy manager may compare the back-end server's computingload to a target load. The target load may be the load at which back-endserver operates with its highest performance to power ratio. An energyefficient point may then be the lowest frequency at which the back-endserver can sustain the target load.

For example, assume that a back-end server has its highest performanceto power ratio when it is operating at a 60% load. If the back-endserver is operating at a 40% load, the resource and energy manager mayuse DVFS to decrease the frequency of the processors. The decreasedfrequency will increase the load, and will result in lower energy usage.If, however, the back-end server is operating at an 80% load, theresource and energy manager may use DVFS to increase the frequency,thereby reducing the load and increasing the performance to power ratio.In some embodiments, a user may configure the target load for theback-end server.

If the back-end server is running at an energy efficient point, theprocess may end. If, however, the back-end server is not running at anenergy efficient point, the resource and energy manager may adjust thePMP to increase efficiency at operation 614.

There are a number of ways that the efficiency may be increased. In someembodiments, the resource and energy manager may use Dynamic Voltage andFrequency Scaling (DVFS) to lower the frequency of the back-end server'sprocessors, or processor cores. In other embodiments, work may beconsolidated onto a subset of the cores, and some cores may be disabled(known as core folding) to decrease the number of active cores runningin the back-end server. In still other embodiments, the resource andenergy manager may put some cores or processor in a low power mode, suchas a nap or sleep mode.

In some embodiments, the frequency may be reduced by a set amount (e.g.100 MHz) in discrete steps at operation 614. After each reduction in thefrequency, the process may resume at operation 604. In otherembodiments, the resource and energy manager may predict the appropriatePMP using, e.g., Little's Law. After the PMP has been adjusted toincrease efficiency at operation 614, the process may end.

In some embodiments, all the operations in method 600 may be performedby the same application on the front-end server. In other embodiments,the operations in method 600 may be performed by two or more servers. Instill other embodiments, the operations may be performed by two or moreapplications running on the front-end server, or by two or more modulesrunning within the same application. For example, the applicationexecuting method 600 may include 3 modules: a service queue module, aQoS monitoring module, and a resource and energy management module.

The service queue module may include instructions to determine thenumber of service queues active in the system, receive a request from adevice, determine the request's service class based on its QoS criteria,and assign the request to the appropriate service queue (i.e. performmethod 500). The QoS monitoring module may include instructions tomonitor the queue depth of each service queue, and instructions todetermine whether the back-end servers are processing the requests fastenough to meet the QoS criteria (i.e. perform operations 602, 604, and606). The resource and energy management module may include instructionsto adjust the power management policies of the back-end servers toensure that the back-end servers are meeting the QoS criteria of theservice queues to which they are assigned, and instructions to ensurethat the back-end servers are running at an energy efficient point giventhe QoS criteria (i.e. perform operations 608, 610, 612, and 614).

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the variousembodiments. As used herein, the singular forms “a,” “an,” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“includes” and/or “including,” when used in this specification, specifythe presence of the stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof. In the previous detaileddescription of exemplary embodiments of the various embodiments,reference was made to the accompanying drawings (where like numbersrepresent like elements), which form a part hereof, and in which isshown by way of illustration specific exemplary embodiments in which thevarious embodiments may be practiced. These embodiments were describedin sufficient detail to enable those skilled in the art to practice theembodiments, but other embodiments may be used and logical, mechanical,electrical, and other changes may be made without departing from thescope of the various embodiments. In the previous description, numerousspecific details were set forth to provide a thorough understanding thevarious embodiments. But, the various embodiments may be practicedwithout these specific details. In other instances, well-known circuits,structures, and techniques have not been shown in detail in order not toobscure embodiments.

Different instances of the word “embodiment” as used within thisspecification do not necessarily refer to the same embodiment, but theymay. Any data and data structures illustrated or described herein areexamples only, and in other embodiments, different amounts of data,types of data, fields, numbers and types of fields, field names, numbersand types of rows, records, entries, or organizations of data may beused. In addition, any data may be combined with logic, so that aseparate data structure may not be necessary. The previous detaileddescription is, therefore, not to be taken in a limiting sense.

The descriptions of the various embodiments of the present disclosurehave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein. Therefore, it is intended that the following claims beinterpreted as covering all such alterations and modifications as fallwithin the true spirit and scope of the invention.

What is claimed is:
 1. A method of handling requests queued to serversbased on a Quality of Service (QoS) criteria, the method comprising:grouping computing resources according to a QoS criteria, wherein thereare at least two groups of computing resources, each group having apower management policy (PMP); receiving a plurality of requests from aplurality of devices; determining a service class for each request;queuing the each request to one of the at least two groups of computingresources based on the service class; and processing the each request bythe one of the at least two groups of computing resources.
 2. The methodof claim 1, wherein there are at least three groups of computingresources with a first group supporting a real-time response and havinga first PMP ensuring the real-time response, a second group supporting adeadline response having a second PMP ensuring the deadline responsewith power saving improvements compared to the first PMP, and a thirdgroup supporting an available resources response having a third PMP withpower saving improvements compared to the second PMP.
 3. The method ofclaim 1, wherein the computing resources are one or more of the groupconsisting of processing cores, central processing units (CPUs), andserver computers.
 4. The method of claim 1, wherein groups of computingresources is a cloud based system with a plurality of separate machineseach having a plurality of processing cores and wherein the plurality ofdevices are mobile devices.
 5. The method of claim 1, furthercomprising: receiving a set of service queue information for a group ofcomputing resources; determining whether the group of computingresources will satisfy the QoS criteria; and adjusting, in response todetermining that the group of computing resources will not satisfy theQoS criteria, the PMP for the group of computing resources.
 6. Themethod of claim 5, wherein the set of service queue informationcomprises a queue depth, and wherein the determining whether the groupof computing resources will satisfy the QoS criteria comprises comparingthe queue depth to a maximum queue depth threshold.
 7. The method ofclaim 5, wherein the set of service queue information comprises a queuedepth and an arrival rate, and wherein determining whether the group ofcomputing resources will satisfy the QoS criteria comprises determiningan average response time for incoming requests.
 8. The method of claim5, wherein adjusting the PMP for the group of computing resourcescomprises using dynamic voltage and frequency scaling (DVFS).
 9. Themethod of claim 5, wherein adjusting the PMP for the group of computingresources comprises consolidating work using processor folding.
 10. Themethod of claim 5, wherein adjusting the PMP for the group of computingresources comprises activating additional cores using processorunfolding.
 11. The method of claim 5, wherein adjusting the PMP for thegroup of computing resources comprises placing one or more of thecomputing resources in a low power mode.
 12. The method of claim 5,wherein adjusting the PMP for the group of computing resources comprisestaking one or more of the computing resources out of a low power mode.13. The method of claim 5, further comprising: determining, in responseto determining that the group of computing resources will satisfy theQoS, whether the group of computing resources is running at an energyefficient point; and adjusting, in response to determining that thegroup of computing resources is not running at an energy efficientpoint, the PMP for the group of computing resources.
 14. The method ofclaim 13, wherein determining whether the group of computing resourcesis running at an energy efficient point comprises: determining acomputing load for the group of computing resources; determining atarget load for the group of computing resources; comparing thecomputing load to the target load.
 15. A computer program product formanaging the energy efficiency of servers providing multi-classcomputing services with QoS assurance, the computer program productcomprising: one or more computer readable storage media and programinstructions stored on at least one of the one or more computer readablestorage media, the program instructions comprising: program instructionsto group computing resources into at least two groups of computingresources, each group of computing resources having a power managementpolicy (PMP), according to a QoS criteria; program instructions toreceive a plurality of requests from a plurality of devices; programinstructions to determine a service class for each request; programinstructions to queue the each request to one of the at least two groupsof computing resources based on the service class; and programinstructions to process the each request by the one of the at least twogroups of computing resources.
 16. The computer program product of claim15, wherein there are at least three groups of computing resources witha first group supporting a real-time response and having a first powermanagement policy (PMP) ensuring the real-time response, a second groupsupporting a deadline response having a second PMP ensuring the deadlineresponse with power saving improvements compared to the first PMP, and athird group supporting an available resources response having a thirdPMP with power saving improvements compared to the second PMP.
 17. Asystem comprising: a memory to store a service queue module, a QoSmonitoring module, and a resource and energy management module; and aprocessor configured to: run the service queue module when a requestwith a QoS criteria is received, wherein the service queue moduledetermines a service class for the request; run the QoS monitoringmodule, wherein the QoS monitoring module is configured to: receive aset of service queue information; determine whether a group of computingresources will meet the QoS criteria; and determine, in response todetermining that the group of computing resources will meet the QoScriteria, whether the group of computing resources are running at anenergy efficient point; run the resource and energy management module,wherein the resource and energy management module is configured to:adjust, in response to the QoS monitoring module determining that thegroup of computing resources will not meet the QoS criteria, a powermanagement policy (PMP) of the group of computing resources to increaseperformance; and adjust, in response to the QoS monitoring moduledetermining that the group of computing resources are not running at anenergy efficient point, the PMP of the set of computing resources toincrease efficiency.
 18. The system of claim 17, the system furthercomprising: a plurality of groups of computing resources configured toprocess the request.